System and method for object recognition using local binarization

ABSTRACT

An object recognition method performed by an object recognition system includes: setting a first binarization area in a portion of a binarization target area of an image, and a first coefficient calculation area including the first binarization area and having an area wider than the first binarization area by a predetermined ratio; determining a first binarization coefficient on the basis of pixel values included in the first coefficient calculation area; performing binarization on the first binarization area through the determined first binarization coefficient; setting a second binarization area in a portion other than the first binarization area, and a second coefficient calculation area including the second binarization area and having an area wider than the second binarization area; determining a second binarization coefficient on the basis of pixel values included in the second coefficient calculation area; and performing binarization on the second binarization area through the determined second binarization coefficient.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority from and the benefit of Korean PatentApplication No. 10-2019-0022779, filed Feb. 26, 2019, which is herebyincorporated by reference for all purposes as if fully set forth herein.

BACKGROUND Field

The present invention relates to an object recognition system and amethod thereof, and more specifically, to a system and a method capableof detecting a position (an outer line) of an object having a specificshape more effectively.

In addition, the present invention relates to an object recognitionsystem and a method thereof, which can detect, through localbinarization, an object robust to an environment in which thecharacteristic of an object greatly affected by illumination or thecharacteristic of an image in each region of a whole image varies.

Discussion of the Background

There is a growing need for recognition of objects existing in variousfields.

Object recognition determines whether an object to be detected exists inan image through object detection and is also required even in a servicefor recognizing meaningful information displayed on a detected object.

For example, in order to rapidly and accurately recognize information tobe recognized from a captured image in a service of recognizing a cardnumber displayed on a financial card (e.g., a credit card or a checkcard, etc.) or a license plate of a vehicle, it may be effective todetect first where a corresponding object is located in the image.

That is, compared to optical character recognition (OCR) performed on animage itself to recognize meaningful information displayed on the image,if OCR is performed in a predetermined method after the position of atarget object is accurately identified, information may be recognizedmore accurately.

Accordingly, it is required to provide a method capable of detecting aposition of an object (an outer line of the object) more effectively.

In addition, when a predetermined recognition target object displayed ona detected object (e.g., text displayed on a financial card) isrecognized, binarization may be performed through a predeterminedpreprocess, and recognition of the recognition target object may beperformed through the binarization, and therefore, it is required toprovide a system and method capable of effectively performing thebinarization.

SUMMARY

Therefore, the present invention has been made in view of the aboveproblems, and it is an object of the present invention to provide amethod capable of accurately detecting an object at a relatively highspeed, and a system thereof.

To accomplish the above object, according to one aspect of the presentinvention, there is provided an object recognition method of recognizinga recognition target object using local binarization, the methodcomprising the steps of: setting a first binarization area in a portionof a binarization target area of an image in the image in which therecognition target object is displayed, and a first coefficientcalculation area including the first binarization area and having anarea wider than the first binarization area by a predetermined ratio, byan object recognition system; determining a first binarizationcoefficient on the basis of pixel values of pixels included in the firstcoefficient calculation area, by the object recognition system;performing binarization on the first binarization area through thedetermined first binarization coefficient, by the object recognitionsystem; setting a second binarization area in a portion other than thefirst binarization area, and a second coefficient calculation areaincluding the second binarization area and having an area wider than thesecond binarization area, by the object recognition system; determininga second binarization coefficient on the basis of pixel values of pixelsincluded in the second coefficient calculation area, by the objectrecognition system; and performing binarization on the secondbinarization area through the determined second binarizationcoefficient, by the object recognition system.

The size of the first binarization area is a size that is set based on asize of the recognition target object.

The step of determining a first binarization coefficient on the basis ofpixel values of pixels included in the first coefficient calculationarea, by the object recognition system, may include the step ofdetermining the first binarization coefficient between an average valueand a maximum value of pixel values of pixels included in the firstcoefficient calculation area, by the object recognition system.

The step of setting a second binarization area in a portion other thanthe first binarization area, and a second coefficient calculation areaincluding the second binarization area and having an area wider than thesecond binarization area, by an object recognition system, includes thestep of setting the second binarization area and the second coefficientcalculation area of a size the same as that of the first binarizationarea and the first coefficient calculation area, by an objectrecognition system.

The object recognition method using local binarization may furthercomprise the steps of: labeling the binarization target area afterbinarization of the binarization target area is completed, by the objectrecognition system; searching for a preset pattern according to adisplay characteristic of the recognition target object from a result ofthe labeling, by the object recognition system; and performingrecognition on a recognition target object included in the searchedpattern.

According to another embodiment, there is provided an object recognitionmethod of recognizing a recognition target object using localbinarization, the method comprising the steps of: a) setting abinarization area of a preset size in a portion of a binarization targetarea of an image in the image in which the recognition target object isdisplayed, by an object recognition system; b) determining abinarization coefficient for the set binarization area, by the objectrecognition system; c) performing binarization on the binarization areausing the determined binarization coefficient, by the object recognitionsystem; and d) moving the binarization area in the binarization targetarea, and performing the steps a), b) and c) on the moved binarizationarea until binarization of the binarization target area is completed, bythe object recognition system.

The object recognition method may be implemented by a computer programinstalled in a data processing device.

To accomplish the above object, according to another aspect of thepresent invention, there is provided an object recognition system forrecognizing a recognition target object using local binarization, thesystem comprising: an area setting module for setting a firstbinarization area in a portion of a binarization target area of an imagein the image in which the recognition target object is displayed, and afirst coefficient calculation area including the first binarization areaand having an area wider than the first binarization area by apredetermined ratio; a coefficient determination module for determininga first binarization coefficient on the basis of pixel values of pixelsincluded in the first coefficient calculation area; and a control modulefor performing binarization on the first binarization area through thedetermined first binarization coefficient, wherein the area settingmodule sets a second binarization area in a portion other than the firstbinarization area, and a second coefficient calculation area includingthe second binarization area and having an area wider than the secondbinarization area; the coefficient determination module determines asecond binarization coefficient on the basis of pixel values of pixelsincluded in the second coefficient calculation area; and the controlmodule performs binarization on the second binarization area through thedetermined second binarization coefficient.

The coefficient determination module may determine the firstbinarization coefficient between an average value of pixel values ofpixels included in the first coefficient calculation area and a maximumvalue among the pixel values.

The area setting module may set the second binarization area and thesecond coefficient calculation area of a size the same as that of thefirst binarization area and the first coefficient calculation area,respectively.

The object recognition system using local binarization may furthercomprise a recognition processing module for labeling the binarizationtarget area after binarization of the binarization target area iscompleted; searching for a preset pattern according to a displaycharacteristic of the recognition target object from a result of thelabeling; and processing recognition on a recognition target objectincluded in the searched pattern.

According to another embodiment, there is provided an object recognitionsystem for recognizing a recognition target object using localbinarization, the system comprising: an area setting module for settinga binarization area of a preset size in a portion of a binarizationtarget area of an image in the image in which the recognition targetobject is displayed; a coefficient determination module for determininga binarization coefficient for the set binarization area; and a controlmodule for performing binarization on the binarization area using thedetermined binarization coefficient, wherein the control module controlsto move the binarization area in the binarization target area, determinea new binarization coefficient for the moved binarization area, andbinarize the moved binarization area using the determined newbinarization coefficient until binarization of the binarization targetarea is completed.

According to the spirit of the present invention, there is an effect ofdetecting an object relatively accurately although the outer line of theobject to be detected is detected broken and extracted not beingclearly.

In addition, since binarization is performed using differentbinarization criteria in each area when the information displayed on anobject is recognized, there is an effect of recognizing well a target tobe recognized even when the target is greatly affected by a lightingenvironment according to a situation of the object or when there is abig difference in the characteristics of background locally.

BRIEF DESCRIPTION OF THE DRAWINGS

To more sufficiently understand the drawings cited in the detaileddescription of the present invention, a brief description of eachdrawing is provided.

FIG. 1 is a view schematically showing the logical configuration of anobject recognition system according to the spirit of the presentinvention.

FIG. 2 is a flowchart schematically illustrating an object detectionmethod according to an embodiment of the present invention.

FIG. 3 is a view showing a region of interest in an object detectionmethod according to an embodiment of the present invention.

FIG. 4 is a view showing an example of extracting line segmentsaccording to an embodiment of the present invention.

FIG. 5 is a view showing a merged line segment according to anembodiment of the present invention.

FIG. 6 is a view showing an example of a detected object according to anembodiment of the present invention.

FIG. 7 is a view showing an example of warping a detected objectaccording to an embodiment of the present invention.

FIG. 8 is a flowchart schematically illustrating a method of recognizinga recognition target object according to an embodiment of the presentinvention.

FIG. 9 is a view showing the concept of performing local binarizationaccording to an embodiment of the present invention.

FIG. 10 is a view showing an example of labeling after performing localbinarization according to an embodiment of the present invention.

FIG. 11 is a view showing a result of searching for a pattern accordingto an embodiment of the present invention.

DETAILED DESCRIPTION

Since the present invention may be diversely converted and have variousembodiments, specific embodiments will be shown in the drawings anddescribed in detail in the detailed description. However, it should beunderstood that this is not intended to limit the present invention tothe specific embodiments, but to comprise all conversions, equivalentsand substitutions included in the spirit and scope of the presentinvention. In describing the present invention, if it is determined thatthe detailed description on the related known art may obscure the gistof the present invention, the detailed description will be omitted.

The terms such as “first” and “second” can be used in describing variousconstitutional components, but the above constitutional componentsshould not be restricted by the above terms. The above terms are usedonly to distinguish one constitutional component from the other.

The terms used herein are used only to describe particular embodimentsand are not intended to limit the present invention. A singularexpression includes a plurality of expressions, unless the contextclearly indicates otherwise.

In this specification, it should be further understood that the terms“include” and “have” specify the presence of stated features, numerals,steps, operations, constitutional components, parts, or a combinationthereof, but do not preclude the presence or addition of one or moreother features, numerals, steps, operations, constitutional components,parts, or a combination thereof.

In addition, in this specification, when any one of constitutionalcomponents “transmits” a data to another constitutional component, itmeans that the constitutional component may directly transmits the datato another constitutional component or may transmit the data to anotherconstitutional component through at least one of the otherconstitutional components. On the contrary, when any one of theconstitutional components directly transmits a data to anotherconstitutional component, it means that the data is transmitted toanother constitutional component without passing through the otherconstitutional components.

Hereinafter, the present invention is described in detail focusing onthe embodiments of the present invention with reference to the attacheddrawings. Like reference symbols presented in each drawing denote likemembers.

FIG. 1 is a view schematically showing the logical configuration of anobject recognition system according to the spirit of the presentinvention.

Referring to FIG. 1, an object recognition system 100 may be implementedto implement an object recognition method according to the spirit of thepresent invention.

The object recognition system 100 (hereinafter, referred to as arecognition system) may detect a desired object from an image. Inaddition, the recognition system 100 may recognize a target desired tobe recognized (hereinafter, referred to as a recognition target object)from the detected object.

The recognition system 100 may be installed in a predetermined dataprocessing system (not shown) to implement the spirit of the presentinvention.

The data processing system means a system having a computing capabilityfor implementing the spirit of the present invention, and averageexperts in the technical field of the present invention may easily inferthat any system capable of performing a service using object detectionaccording to the spirit of the present invention, such as a personalcomputer, a portable terminal, or the like, as well as a network servergenerally accessible by a client through a network, may be defined asthe data processing system defined in this specification.

Hereinafter, although a case in which the object to be detected is afinancial card (e.g., a credit card, a check card, etc.) and therecognition target object is a numeral or text such as a card number oran expiry date printed or embossed on the financial card is described asan example in the present specification, average experts in thetechnical field of the present invention may easily infer that thetechnical spirit of the present invention may be applied to any objecthaving a predetermined shape (e.g., a rectangle or the like).

The data processing system may include a processor and a storage device.The processor may mean a computing device capable of driving a programfor implementing the spirit of the present invention, and the processormay perform a function defined in this specification by driving theprogram.

The storage device may means a data storage means capable of storing theprogram, and may be implemented as a plurality of storage meansaccording to embodiments. In addition, the storage device may mean notonly a main memory device included in the data processing system, butalso a temporary storage device or a memory that can be included in theprocessor.

Although it is shown in FIG. 1 that the recognition system 100 isimplemented as any one physical device, average experts in the technicalfield of the present invention may easily infer that a plurality ofphysical devices may be systematically combined as needed to implementthe recognition system 100 according to the spirit of the presentinvention.

According to the spirit of the present invention, the recognition system100 may detect a corresponding object from an image in which an objectis displayed according to the spirit of the present invention. Detectingan object may mean detecting a position of the object from the image,and in this specification, it may mean extracting outer linesconfiguring the object.

In addition, the recognition system 100 may recognize an object, whichis a recognition target displayed in the object, i.e., a recognitiontarget object.

The recognition system 100 implemented for this function may have aconfiguration as shown in FIG. 1.

The recognition system 100 may include a control module 110, anextraction module 120, and a merge module 130. The recognition system100 may further include a preprocessing module 140. According toembodiments, the recognition system 100 may further include an areasetting module 150, a coefficient determination module 160, and/or arecognition processing module 170. According to another embodiment, therecognition system 100 may include only the control module 110, the areasetting module 150, and the coefficient determination module 160, andmay further include the recognition processing module 170 as needed.

The recognition system 100 may means a logical configuration havinghardware resources and/or software needed for implementing the spirit ofthe present invention, and does not necessarily means a physicalcomponent or a device. That is, the recognition system 100 may mean alogical combination of hardware and/or software provided to implementthe spirit of the present invention, and if necessary, the recognitionsystem 100 may be installed in devices spaced apart from each other andperform respective functions to be implemented as a set of logicalconfigurations for implementing the spirit of the present invention. Inaddition, the recognition system 100 may mean a set of componentsseparately implemented as each function or role for implementing thespirit of the present invention. For example, each of the control module110, the extraction module 120, the merge module 130, the preprocessingmodule 140, the area setting module 150, the coefficient determinationmodule 160 and/or the recognition processing module 170 may be locatedin different physical devices or in the same physical device. Inaddition, according to embodiments, combinations of software and/orhardware configuring each of the control module 110, the extractionmodule 120, the merge module 130, the preprocessing module 140, the areasetting module 150, the coefficient determination module 160 and/or therecognition processing module 170 may also be located in differentphysical devices, and components located in different physical devicesmay be systematically combined with each other to implement each of theabove modules.

In addition, a module in this specification may mean a functional andstructural combination of hardware for performing the spirit of thepresent invention and software for driving the hardware. For example,average experts in the technical field of the present invention mayeasily infer that the module may mean a logical unit of a predeterminedcode and hardware resources for performing the predetermined code, anddoes not necessarily mean a physically connected code or a kind ofhardware.

The control module 110 may control the components (e.g., the extractionmodule 120, the merge module 130, the preprocessing module 140, the areasetting module 150, the coefficient determination module 160 and/or therecognition processing module 170) included in the recognition system100 or manage their functions and/or resources to implement the spiritof the present invention.

The recognition system 100 may know in advance a shape of an object tobe detected. In addition, the recognition system 100 may detect anobject of the shape.

Hereinafter, although a case in which the object is a financial card isdescribed as an example in this specification, average experts in thetechnical field of the present invention may easily infer that thespirit of the present invention may be used to detect various objectshaving a predetermined shape.

The extraction module 120 may extract line segments from an image. Theshape of the object is set in the extraction module 120 in advance, andsince the object may be a rectangle according to embodiments, theboundary of the object may be a straight line. Therefore, the extractionmodule 120 may extract line segments that may be all or part of theouter line, which are straight lines forming the boundary of the object,from the image.

The method of extracting the line segments from the image may vary. Forexample, edges displayed in the image may be detected through edgedetection, and line segments may be extracted by extracting non-curvedlines among the detected edges. Some of the extracted line segments maybe all or part of the outer line, and the line segments extractedaccording to the image features displayed in the object, not the outerline, or the line segments extracted by the image features existingoutside the object may be included in the extracted line segments.

In addition, predetermined preprocessing may be performed on the imagephotographed by an image capturing apparatus to extract these linesegments more effectively.

For example, the preprocessing module 140 may separate channels for eachchannel of color (e.g., R, G, B or y, cb, cr, etc.) in the originalimage photographed by the image capturing apparatus. In addition,according to embodiments, the preprocessing module 140 may furtherperform predetermined filter processing. Then, the extraction module 120may extract line segments from any one or a plurality of preprocessedimages.

On the other hand, since the object has a predetermined shape and theposition of each of the outer lines may be limited according to theshape, the extraction module 120 may extract line segments from eacharea, in which the outer lines are likely to be located, for moreeffective and faster detection of the object. The region that is setlike this will be defined as a region of interest in this specification.

For example, when the object is a rectangular financial card, the outerline of the financial card may have an upper side, a lower side, a leftside, and a right side. A corresponding region of interest may beassigned to each of the outer lines.

When a region of interest is assigned like this and the line segmentsare extracted from each region of interest or the line segments aremerged in each region of interest as described below, the object may bedetected within a shorter time. This is since that the direction of theouter line may be specified in advance for each region of interest.

FIG. 3 is a view showing a region of interest in an object detectionmethod according to an embodiment of the present invention, and when theobject is a financial card as shown in FIG. 3, four regions of interest11, 12, 13 and 14 may be set from the image 10. Each of the regions ofinterest 11, 12, 13 and 14 may be a region in which each of the outerlines of the financial card may exist. Of course, the regions ofinterest 11, 12, 13 and 14 may be set as regions having a suitable sizeso that at least an outer line may be included.

According to an embodiment, the extraction module 120 may extract linesegments only from the set regions of interest 11, 12, 13 and 14, or mayextract line segments from the entire image and select only the segmentsincluded in the regions of interest 11, 12, 13 and 14. According toembodiments, the extraction module 120 may extract line segments fromthe entire image, and the merge module 130 may select only the linesegments belonging to the regions of interest 11, 12, 13 and 14 amongthe extracted line segments and use the selected segments as a target ofmerge.

In any case, each of the line segments extracted from the image may bemanaged to confirm to which region of interest 11, 12, 13, and 14 theline segments belong.

For example, FIG. 4 is a view showing an example of extracting linesegments according to an embodiment of the present invention. As shownin FIG. 4, the extraction module 120 may extract line segmentsseparately for each of the regions of interest 11, 12, 13 and 14, andthe extracted line segments may be as shown in FIG. 4

Then, the merge module 130 included in the recognition system 100 maygenerate a merged line segment on the basis of the extracted linesegments. The merge module 130 may generate merged line segments on thebasis of the directionality of each of the extracted line segments.

In addition, the merge module 130 may generate a merged line segment foreach region of interest. Generating a merged line segment for eachregion of interest means that a merged line segment corresponding to anyone region of interest (e.g., a first region of interest 11) isgenerated by merging only the line segments extracted from the region ofinterest (e.g., the first region of interest 11).

The reason why the merge module 130 generates a merged line segment fromthe extracted line segments is that a case in which one outer line of anobject is cut into a plurality of pieces and detected as a plurality ofline segments is more frequent than a case in which the outer line ofthe object is wholly extracted as a single line segment according to thestate or the photographing environment of an image. Accordingly,generating a merged line segment may be to find out which line segmentsamong the line segments extracted or selected for each of the regions ofinterest 11, 12, 13 and 14 are the line segments corresponding to theouter line.

For example, as shown in FIG. 4, it is understood that all the upperside, the lower side, the left side, and the right side of the financialcard are not detected as a line segment, but each one side is extractedas a plurality of broken line segments.

Meanwhile, since the directionality of an outer line corresponding toeach of the regions of interest 11, 12, 13, and 14 is already determinedbefore a merged line segment is generated, the merging module 130 mayexclude line segments, having a big difference in the directionalitywith an outer line of a corresponding region of interest among theextracted line segments, from the target of merge. According toembodiments, the extraction module 120 may fundamentally delete the linesegments having a big difference from the direction of the outer linescorresponding to the regions of interest 11, 12, 13 and 14 from theextracted line segments.

The merge module 130 and/or the extraction module 120 may exclude linesegments having a predetermined or larger slope compare to direction ofan outer line to i.e., directionality corresponding to each of theregions of interest 11, 12, 13 and 14, from the target of merge orfundamentally delete them from the extracted segments. For example, theouter lines corresponding to the first region of interest 11 and thethird region of interest 13 may be the upper side and the lower side andhave a direction close to the horizontal line although they areprojected onto the camera plane. In this case, among the line segmentsextracted from the first region of interest 11 and the third region ofinterest 13, line segments inclined more than a predetermined angle(e.g., 30 degrees, 45 degrees, etc.) from the horizontal line may beexcluded from the target of merge or fundamentally deleted from the listof extracted line segments.

In addition, the outer lines corresponding to the second region ofinterest 12 and the fourth region of interest 14 may be the right sideand the left side of the financial card and have a direction close tothe vertical line although they are projected onto the camera plane. Inthis case, among the line segments extracted from the second region ofinterest 12 and the fourth region of interest 14, the line segmentsinclined more than a predetermined angle (e.g., 30 degrees, 45 degrees,etc.) from the vertical line may be excluded from the target of merge orfundamentally deleted from the list of extracted line segments.

Then, the merge module 130 may generate a merged line segment for eachof the regions of interest 11, 12, 13 and 14.

An example of generating the merged line segment by the merge module 130will be described in detail with reference to FIG. 5.

FIG. 5 is a view showing a merged line segment according to anembodiment of the present invention.

Referring to FIG. 5, the merging module 130 may merge line segmentshaving a directionality satisfying a reference condition among the linesegments remaining in each region of interest, i.e., the line segmentsremaining after excluding the line segments excluded from the target ofmerge since there is a big difference from the directionality of acorresponding region of interest.

The reference condition may be a directionality the same or similar asmuch as to satisfy a predetermined reference condition among the linesegments remaining in each of the regions of interest 11, 12, 13 and 14.

For example, when the merge module 130 generates a merged line segmentfrom the first region of interest 11, as shown in FIG. 5, it may selectany one line segment (a), i.e., a reference line segment, among the linesegments presently remaining in the first region of interest 11. Forexample, a line segment having a directionality most similar to thedirectionality (e.g., a horizontal line) of a corresponding region ofinterest 11, i.e., closest to the horizontal line, may be selected firstas the reference line segment (a). Of course, according to embodiments,the merging process may be performed by sequentially setting referenceline segments for all or some of the line segments.

Then, the merge module 130 may select other line segments of the firstregion of interest 11 which satisfy a predetermined condition indirectionality with the reference line segment (a).

For example, in order to determine whether the directionality satisfiesa predetermined condition, an angle formed by the reference line segment(a) or the extension line (a′) of the reference line segment (a) and theother line segment may be used. In this case, a condition for crossingthe extension line (a′) of the reference line segment (a) and the otherline segment should be added, or a condition related to distance mayneed to be additionally defined.

Alternatively, when the orthogonal distances between the extension line(a′) of the reference line segment (a) and both end points (e.g., bp1and bp2) of the other line segment (e.g., b) are smaller than or equalto a predetermined threshold value, respectively, it may be determinedthat the other line segment satisfies the predetermined condition.

In the embodiment of FIG. 5, line segment b and line segment c may beline segments satisfying the directionality condition with the referenceline segment (a), and line segment d and line segment e may be linesegments that do not satisfy the condition.

Then, reference line segment (a), line segment b, and line segment c maybe line segments that can be merged.

The merged line segment may be the sum of the lengths of reference linesegment (a) and each length of portion of reference line segment's (a)direction of other line segment b and c, and the direction of mergedline segment may be same of the reference line segment (a).

That is, each length of portion of reference line segment's (a)direction of other line segment b and c may be the lengths of projectedportion obtained by projecting each other line segments (e.g., b and c)to the extension line (a′) of the reference line segment (a).

Then, the generated merged line has the direction of the reference linesegment (a), and the length may be the sum of the length of the linesegment (a), the projection length with respect to the extension line(a′) of line segment b, and the projection length with respect to theextension line (a′) of line segment c.

According to an embodiment, the merge module 130 may generate at leastone merged line segment in the first region of interest 11 by changingthe reference line segment while maintaining the merged line segmentwithout deleting it from the line segment list of the first region ofinterest 11.

In this manner, the merge module 130 may generate at least one mergedline segment for each of the regions of interest 11, 12, 13 and 14.Then, the segment sets in which the merged segments and the originalsegments are maintained for each of the regions of interest 11, 12, 13and 14 may be maintained.

Thereafter, the control module 110 may extract the line segments thatmay be all or part of the actual outer line of the object one by one foreach of the regions of interest 11, 12, 13 and 14. That is, the linesegments may be extracted one by one from the segment set maintained foreach of the regions of interest 11, 12, 13 and 14. The extracted linesegment may be all or part of the outer line of each region of interest.

In this case, since the longest line segment among the set of segmentsis likely to be all or part of an actual outer line, it may be effectivefor the control module 110 to sequentially extract the line segments inorder of length from the segment sets of each of the regions of interest11, 12, 13 and 14.

In addition, since even the longest merged line segment may be shorterthan the length of the outer line of the actual object, the controlmodule 110 may specify the outer line of the shape formed by theextension lines of the line segments extracted from the regions ofinterest 11, 12, 13 and 14, i.e., the outer line of a candidate figure,as a candidate outer line. For example, the longest line segments areextracted one by one from each of the regions of interest 11, 12, 13 and14, and a candidate outer line is specified on the basis of theextracted line segments, and if it is determined that the candidateouter line is not an outer line of the actual object, a process ofsequentially extracting the next longest line segment and specifying acandidate outer line may be repeated while changing the region ofinterest. Of course, the order and/or the method of extracting a linesegment to specify a candidate outer line for each of the regions ofinterest 11, 12, 13 and 14 may be diverse according to embodiments, anda line segment may be extracted from each of the regions of interest 11,12, 13 and 14 in various ways, and an outer line of a figure formed byan extension line of the extracted line segment may be specified as acandidate outer line.

An example of a specific candidate outer line may be as shown in FIG. 6.

FIG. 6 is a view showing an example of a detected object according to anembodiment of the present invention.

FIG. 6 shows a case in which a specific candidate outer line is an outerline of an actual object, and the specified candidate outer lines 20,21, 22 and 23 may be extension lines of the line segments extracted fromthe regions of interest 11, 12, 13 and 14, respectively. In addition, afigure formed by the extension lines may be a candidate figure as shownin FIG. 6.

Then, the control module 110 may determine whether the appearance of thecandidate figure corresponds to the appearance attribute of the objectto be detected. That is, it may be determined whether specific candidateouter lines 20, 21, 22 and 23 correspond to the appearance attributes ofthe object.

For example, in the case of a financial card, the appearance attributemay be a predetermined aspect ratio (for example, 1.5858:1 in the caseof ISO7810), and the control module 110 may determine whether a figureformed by the specific candidate outer lines conforms to the appearanceattribute.

If it is determined that the candidate figure, i.e., the candidate outerlines, corresponds to the appearance attribute of the object,corresponding candidate outer lines may be determined as the outer linesof the actual object.

In this case, since the candidate outer lines are line segmentsextracted from the object in a state projected on the camera plane,although the candidate outer lines are the outer line of the actualobject, the length of the outer line may be distorted. For example,although the object is a rectangle and the candidate outer lines are theouter lines of the actual object, the outer lines of the object in astate projected onto the camera plane may not be a rectangle.

Accordingly, it needs to adjust the distortion to more accuratelydetermine whether a candidate figure or candidate outer lines correspondto the appearance attribute (e.g., a rectangle having a fixed aspectratio) of the actual object.

That is, the control module 110 may use the length ratio of thecandidate outer lines as the appearance attribute to determine whetherthe current candidate outer lines correspond to the appearance attributeof the object.

In this case, the length ratio of the candidate outer lines may be aratio of length that the respective outer lines have with respect to allthe candidate outer lines constituting the candidate figure, or a ratioof length that some (e.g., any one of the horizontal sides and any oneof the vertical sides) of the candidate outer lines have. Alternatively,it may be a ratio that a value should have after a predeterminedoperation is performed on at least some of the candidate outer lines. Inany case, the term “length ratio” herein is calculated on the basis ofthe length of a line segment (outer line) of a specific figure and maymean a unique value of the specific figure.

For example, as described above, when the object is a financial card,the financial card may have a ratio of 1.5858:1 as a ratio of ahorizontal side to a vertical side. Accordingly, the length ratio of thecandidate outer lines may also be defined as any value if the value mayconfirm whether the ratio of length between the candidate outer line andthe horizontal outer line corresponds to the ratio of the horizontalside to the vertical side of the financial card.

For example, in this specification, it may be determined whether the sumof length of the horizontal outer lines and the sum of length of thevertical outer lines among the candidate outer lines corresponds to thelength ratio of the financial card. The correspondence may be determinedas a case in which the difference between the ratio of the sum of lengthof the horizontal outer lines to the sum of length of the vertical outerlines among the candidate outer lines and the length ratio of thefinancial card is equal to or smaller than a predetermined thresholdvalue.

In this case, length ratio of the financial card may also be defined asthe ratio of the sum of the horizontal outer lines to the sum of thevertical outer lines, and when the object is rectangular, the lengthratio may be a value equal to the ratio of the length of any one ofpredetermined horizontal outer lines to the length of any one ofpredetermined vertical outer lines. However, since a predetermineddistortion occurs while the candidate outer lines are projected on thecamera plane as described above, the two horizontal outer lines (orvertical outer lines) may not have the same length, and thus it may beeffective to use the ratio of the sum of length of the two horizontalouter lines (or vertical outer lines) as the length ratio.

In addition, since the calculation speed of accurate inverse projectionis very slow, there is an effect of calculating in a short time in thecase of calculating an approximate value using the sum of length of thehorizontal outer lines and the vertical outer lines.

In order to compensate for the distortion of an object on the cameraplane by using the approximation value, the angle of vertices (e.g.,four corners) of the candidate figure may be used.

In the case where the original object is rectangular, it is normal whenthe internal angle of a vertex is 90 degrees, and the greater thedifference from the 90 degrees, the greater the distortion of length ofthe outer lines connected to the vertex.

Accordingly, the control module 110 may correct the length of at leastone of the outer lines on the basis of the difference between theinternal angle (e.g., 90 degrees) that the vertex of a specific figure(e.g., rectangle) of the original object should have and the actualinternal angle of the candidate figure, and calculate a value of thelength ratio on the basis of the corrected length.

An example of this may be as follows.

For example, sum_w may be defined as a sum of two horizontal outer lines(e.g., 20 and 22) among the candidate outer lines.

In addition, sum_h may be defined as a sum of two vertical outer lines(e.g., 21 and 23) among the candidate outer lines.

In addition, aver_angle_error may be an average of the difference valuebetween the angles of four vertices and a value (e.g., 90 degrees) thatthe internal angle of the vertex of the original object should have.

In addition, diff_w may be a difference value between two horizontalouter lines (e.g., 20 and 22) among the candidate outer lines, anddiff_h may be a difference value between two vertical outer lines (e.g.,21 and 23) among the candidate outer lines.

Then, the length ratio corrected on the basis of the angles of fourvertices may be defined as follows.card_wh_ratio={sum_w×(1−sin(aver_angle_error×diff_w_ratio))}/{sum_h×(1−sin(aver_angle_error×diff_h_ratio))}  [Equation1]

Here, diff_w_ratio is diff_w/(diff_w+diff_h), diff_h_ratio may bedefined as 1−diff_w_ratio.

When the corrected length ratio is within a predetermined thresholdvalue from the length ratio 1.5858/1 of the financial card, it may bedetermined that the candidate outer lines correspond to the appearanceattribute of the financial card.

Then, the candidate outer lines may be detected as the outer lines ofthe object. That is, a candidate figure may be detected as an object (ora position of the object).

When the candidate outer lines do not correspond to the appearanceattribute of the object, the control module 110 may detect an objectuntil the object is detected or by repeating a predetermined number oftimes while changing the candidate outer lines.

When an object is detected, according to embodiments, the control module110 may warp the distorted object to have an appearance attribute thatthe object should have originally. An example of the warping may be asshown in FIG. 7.

After warping is performed like this, an application service using thedetected object can be more effectively performed.

For example, since the features of an object exist precisely at thenormalized positions without distortion, faster and more accurateperformance may be achieved in recognizing the features (e.g., a cardnumber, an expiry date, etc.) displayed in the object.

The method of detecting an object according to the spirit of the presentinvention described above may be summarized overall as shown in FIG. 2.

FIG. 2 is a flowchart schematically illustrating an object detectionmethod according to an embodiment of the present invention.

Referring to FIG. 2, the recognition system 100 according to the spiritof the present invention may extract line segments from an image inwhich an object is displayed (S110). In this case, the recognitionsystem 100 may set regions of interest 11, 12, 13 and 14 of the image,and extract line segments for each of the set regions of interest 11,12, 13 and 14 as described above (S100). Of course, according toembodiments, the line segment may be extracted from the whole image, andonly the line segments belonging to the regions of interest 11, 12, 13and 14 may be left among the extracted line segments.

Then, the recognition system 100 may generate merged line segments forthe regions of interest 11, 12, 13 and 14 (S120). Then, a segment setincluding the generated merged line segments may be maintained for eachof the regions of interest 11, 12, 13 and 14.

Then, the recognition system 100 may extract line segments from each ofthe regions of interest 11, 12, 13 and 14, and specify a candidatefigure and/or candidate outer lines formed by the extension lines of theextracted line segments (S130).

Then, it may be determined whether the specific candidate figure and/orcandidate outer lines are an object to be detected (S140). To this end,whether the candidate figure and/or the candidate outer linescorresponds to the appearance attribute of the object may be determined,and in this case, the length ratio of the candidate outer lines may beused. In addition, as described above, the length ratio may becompensated according to the distortion degree of the internal angle ofthe vertex.

In addition, if it is determined that the candidate outer lines are anobject to be detected, the candidate outer lines are determined as theouter lines of the object, and the object detection process may beterminated (S150). If it is determined that the candidate outer linesare not an object to be detected, the process of re-setting thecandidate outer lines and re-determining whether the re-set candidateouter lines are an object to be detected (S140) may be repeated.

Meanwhile, when detection of the object is completed, the recognitionsystem 100 may recognize meaningful information or a target displayed onthe object. In this specification, a target to be recognized is definedas a recognition target object. For example, the recognition targetobject may be meaningful text such as a card number, an expiry date, ora name displayed on the financial card.

According to the spirit of the present invention, the recognition system100 may perform local binarization to recognize a recognition targetobject. The binarization may mean a process of changing a pixel value toa value of 0 or 1, and it is known that when the binarization isperformed, a recognition target object can be recognized with higheraccuracy.

However, in the case where the binarization is uniformly performed onthe entire detected object, when the image feature changes muchaccording to the lighting environment, or only a specific area has acharacteristic different from the image feature of the entire image(e.g., a case in which a background pattern does not exist in thespecific area although a background pattern exists in other areas),there may be a problem in which pixels displaying the meaningfulinformation disappear. For example, when the object is a financial card,and the recognition target object is a card number displayed on thefinancial card, and the card number is embossed, the reflectioncharacteristic of light may be different from those of the other areasonly in the embossed portion of the card number according to thelighting, and in this case, if binarization is performed on the entireobject, the portion of card number may be determined as unnecessarypixels.

Therefore, the recognition system 100 according to the spirit of thepresent invention may perform local binarization.

According to the spirit of the present invention, the recognition system100 may perform binarization on a binarization target area in a detectedobject. Although the binarization target area may be the entire detectedobject, only a part of the object may be set as the binarization targetarea when there is a predetermined condition such that a recognitiontarget object exists only at a specific position within the object.

According to an embodiment of the present invention, the recognitionsystem 100 does not binarize the entire binarization target area usingany one binarization coefficient, but sets a portion of the region,determines a binarization coefficient to be used for the set region, andperforms binarization. Thereafter, the recognition system 100 maydetermine a new binarization coefficient and perform binarization usingthe determined new binarization coefficient while moving the set region.That is, the binarization coefficient which becomes a criterion ofbinarization may vary in each area, and therefore, incorrectbinarization may be prevented considerably according to lightingcharacteristics or regional characteristics.

This concept will be described in detail with reference to FIG. 8.

FIG. 8 is a flowchart schematically illustrating a method of recognizinga recognition target object according to an embodiment of the presentinvention.

Referring to FIG. 8, the area setting module 150 included in therecognition system 100 may set a binarization area from an imagecorresponding to an object (S200). As described above, the binarizationarea may be set in a portion of a binarization target area predefined inthe image.

The image may be, for example, an image in which only a preselectedchannel (e.g., a gray channel) is separated from a camera preview image.The preprocessing module 140 may separate the preselected channel fromthe original image of the detected object and use the separated channelimage for object recognition. In addition, when the recognition targetobject is embossed text (including numerals), i.e., embossing text,binarization may not be easy in general due to the three-dimensionalshape. In addition, since it may be diversely affected by lighting dueto the three-dimensional shape, the preprocessing module 140 may performobject recognition as described below after applying a predeterminedfilter (e.g., Sobel, Scharr, etc.) so that the lighting effect and theembossing text may be expressed well in the image of the separatedchannel.

Then, the area setting module 150 may set a coefficient calculation areaon the basis of the set binarization area (S210).

According to an embodiment, the coefficient calculation area may be setto include the binarization area and have an area wider than thebinarization area by a predetermined ratio.

According to an embodiment, the binarization area may be determinedaccording to the size of the recognition target object. For example,when the recognition target object is a numeral or text and the size ofthe numeral or text is fixed in advance, the binarization area may beset to have an area equal to or larger than the size of the numeral ortext by a predetermined ratio. In addition, according to the shape ofthe recognition target object, the shape of the binarization area may beset to correspond thereto.

Then, the coefficient calculation area may be set to also include thebinarization area and may have the same shape as that of thebinarization area.

An example of the binarization area and the coefficient calculation areamay be as shown in FIG. 9.

FIG. 9 is a view showing the concept of performing local binarizationaccording to an embodiment of the present invention.

As shown in FIG. 9, the area setting module 150 may set a binarizationarea (a) 30 in an area of the binarization target area. The area settingmodule 150 may set a coefficient calculation area (b) 31 including thebinarization area 30 and wider than the binarization area 30 by apredetermined ratio.

The reason why the coefficient calculation area is set to be wider thanthe binarization area like this is that there is a risk in that when thebinarization coefficient is determined based only on the binarizationarea, binarization may be discontinued between areas since abinarization result of an image feature existing across the binarizationarea and an immediately adjacent area is different when differentbinarization coefficients are determined in the binarization area andthe immediately adjacent area. Accordingly, when the coefficientcalculation area is set to have an area wider than the binarizationarea, there is an effect of solving this problem by commonly consideringthe coefficient calculation area when a binarization coefficient of aspecific binarization area is calculated and when a binarizationcoefficient of an area immediately adjacent to the specific binarizationarea is calculated.

Of course, the binarization coefficient may also be determined usingonly the pixels existing in the binarization area according to thecharacteristics of the object or the characteristics of the recognitiontarget object, or according to embodiments. In this case, the areasetting module 150 only needs to set the binarization area, and it isnot needed to set the coefficient calculation area.

Although it is shown in FIG. 9 as an example that the coefficientcalculation area 31 has a center the same as that of the binarizationarea 30, a shape the same as that of the binarization area 30, and anarea to be larger by a predetermined ratio (e.g., 10%, 20%, etc.),various modifications can be made according to embodiments.

Then, the coefficient determination module 160 may determine abinarization coefficient that will be used as a reference for performingbinarization on the binarization area 30, on the basis of the pixelvalues included in the coefficient calculation area 31 (S220).

According to an example, the binarization coefficient may be determinedbetween an average value and a maximum value of pixel values included inthe coefficient calculation area 31.

For example, the binarization coefficient may be calculated by thefollowing equation.Binarization coefficient=Average value+(Maximum value−Averagevalue)×Constant

Here, the constant is a value between 0 and 1 and may be diverselydetermined according to embodiments.

Then, the control module 110 may binarize the binarization area 30 usingthe determined binarization coefficient (S230). For example, a pixelvalue smaller than the binarization coefficient may be set to 0, and apixel value greater than or equal to the binarization coefficient may beset to 1.

Thereafter, the control module 110 determines whether binarization ofthe binarization target area is completed (S240), and if not, thecontrol module 110 controls the area setting module 150 to move theposition of the binarization area 30 (S250). Of course, the movedposition may be also within the binarization target area.

For example, the area setting module 150 may set a new binarization areaby moving the binarization area as much as the horizontal length or thevertical length of the binarization area 30 from the initial position ofthe binarization area 30. The coefficient calculation area is set asdescribed above for the newly set binarization area, and the coefficientdetermination module 160 determines a binarization coefficient, and thecontrol module 110 may perform binarization on the newly setbinarization area using the determined binarization coefficient. Ofcourse, the new binarization area and coefficient calculation area mayhave the same size and shape as those of the previous binarization areaand coefficient calculation area. Of course, the method of determiningthe binarization coefficient may be the same although the binarizationarea is moved.

The binarization may be performed until binarization of the binarizationtarget area is completed while repeatedly moving the binarization areain this manner.

When binarization of the binarization target area is completed, therecognition processing module 170 may perform recognition of therecognition target object using a binarized result image, i.e., an imageof the binarized binarization target area (S260). Of course, thebinarization target area may be the entire object.

The method of recognizing a recognition target object existing in thebinarization target area may be diverse.

According to an embodiment of the present invention, the recognitionprocessing module 170 may perform labeling (e.g., connected componentlabeling) on the binarization target area on which binarization has beenperformed.

The labeling means a process of grouping pixels having the same pixelvalue in a binarized image, and the connected component labeling maymean an algorithm for grouping connected pixels having the same pixelvalue into one object. The things existing in the binarization targetarea may be extracted through the labeling.

An example like this may be as shown in FIG. 10.

FIG. 10 is a view showing an example of labeling after performing localbinarization according to an embodiment of the present invention.

FIG. 10 exemplarily shows an object in which a binarization target areais detected object, i.e., an entire financial card, and exemplarilyshows a result of labeling the binarization target region binarizedthrough local binarization as described above.

In addition, the things extracted as a result of the labeling may bethose indicated as a box as shown in FIG. 10.

Then, the recognition processing module 170 may search for a uniquepattern of the recognition target object to be recognized on the basisof the location of the extracted things.

For example, when the recognition target object is a card number, eachof the numerals included in the card number may have a displaycharacteristic in which the numerals included in the card number are thesame or similar in size and displayed in a uniform shape. A set of thethings having such a characteristic may be the pattern.

Then, the recognition processing module 170 may search for thischaracteristic pattern from the extracted things.

The result of performing the pattern searched like this may be as shownin FIG. 11.

FIG. 11 is a view showing a result of searching for a pattern accordingto an embodiment of the present invention.

Referring to FIG. 11, the patterns 40, 41, 42 and 43 searched by therecognition processing module 170 may be as shown in FIG. 11.

When at least one pattern is searched in this manner, the recognitionprocessing module 170 may perform recognition (e.g., OCR) on thesearched pattern. The recognition processing module 170 may perform theOCR function by itself, or may input the pattern into a separatelyprovided recognition means (not shown) having a OCR function to obtain aresult according to the input, i.e., a recognition result.

In addition, the recognition processing module 170 may input the patternitself into the recognition means, or may input each of the things,i.e., numerals, included in the pattern into the recognition means.Various embodiments can be made depending on the design of therecognition means.

Meanwhile, although a case of setting a coefficient calculation areawhen local binarization is performed has been described in the aboveembodiment, according to embodiments, it does not need to necessarilyset the coefficient calculation area.

Then, the area setting module 150 may set only a binarization areahaving a preset size in a portion of a binarization target area of animage in the image in which a recognition target object is displayed.

The coefficient determination module 160 may determine a binarizationcoefficient for the binarization area described above on the basis ofthe values of the pixels included in the set binarization area.

Then, after performing binarization on the binarization area using thedetermined binarization coefficient, the control module 110 may controlthe area setting module 150 and the coefficient determination module 160to move the binarization area and determine a new binarizationcoefficient for the moved binarization area, and perform binarization onthe moved binarization area using the determined new binarizationcoefficient until binarization of the binarization target area iscompleted.

In any case, according to the spirit of the present invention, asbinarization is performed by adaptively selecting a binarizationcriterion for some areas, instead of performing binarization for entireobjects on the basis of a single uniform criterion, there is an effectof having object recognition accuracy robust in an environment having adifferent image feature only in a specific area.

The object recognition method according to an embodiment of the presentinvention can be implemented as a computer-readable code in acomputer-readable recording medium. The computer-readable recordingmedium includes all kinds of recording devices for storing data that canbe read by a computer system. Examples of the computer-readablerecording medium are ROM, RAM, CD-ROM, a magnetic tape, a hard disk, afloppy disk, an optical data storage device and the like. In addition,the computer-readable recording medium may be distributed in computersystems connected through a network, and a code that can be read by acomputer in a distributed manner can be stored and executed therein. Inaddition, functional programs, codes and code segments for implementingthe present invention can be easily inferred by programmers in the art.

While the present invention has been described with reference to theembodiments shown in the drawings, this is illustrative purposes only,and it will be understood by those having ordinary knowledge in the artthat various modifications and other equivalent embodiments can be made.Accordingly, the true technical protection range of the presentinvention should be defined by the technical spirit of the attachedclaims.

The invention claimed is:
 1. An object recognition method of recognizinga recognition target object using local binarization, the methodcomprising: setting a first binarization area in a portion of abinarization target area of an image in the image in which therecognition target object is displayed, and a first coefficientcalculation area including the first binarization area and having anarea wider than the first binarization area by a predetermined ratio, byan object recognition system; determining a first binarizationcoefficient based on pixel values of pixels included in the firstcoefficient calculation area, by the object recognition system;performing binarization on the first binarization area through thedetermined first binarization coefficient, by the object recognitionsystem; setting a second binarization area in a portion other than thefirst binarization area, and a second coefficient calculation areaincluding the second binarization area and having an area wider than thesecond binarization area, by the object recognition system; determininga second binarization coefficient based on pixel values of pixelsincluded in the second coefficient calculation area, by the objectrecognition system; and performing binarization on the secondbinarization area through the determined second binarizationcoefficient, by the object recognition system.
 2. The method accordingto claim 1, wherein a size of the first binarization area is a size thatis set based on a size of the recognition target object.
 3. The methodaccording to claim 1, wherein the determining a first binarizationcoefficient based on pixel values of pixels included in the firstcoefficient calculation area, by the object recognition system, includesdetermining the first binarization coefficient between an average valueand a maximum value of pixel values of pixels included in the firstcoefficient calculation area, by the object recognition system.
 4. Themethod according to claim 1, wherein the setting a second binarizationarea in a portion other than the first binarization area, and a secondcoefficient calculation area including the second binarization area andhaving an area wider than the second binarization area, by an objectrecognition system, includes setting the second binarization area andthe second coefficient calculation area of a size the same as that ofthe first binarization area and the first coefficient calculation area,by an object recognition system.
 5. The method according to claim 1,further comprising: labeling the binarization target area afterbinarization of the binarization target area is completed, by the objectrecognition system; searching for a preset pattern according to adisplay characteristic of the recognition target object from a result ofthe labeling, by the object recognition system; and performingrecognition on a recognition target object included in the searchedpattern.
 6. A non-transitory computer-readable storage medium installedin a data processing device and storing processor-executable instructionto perform the method-of claim
 1. 7. An object recognition system forrecognizing a recognition target object using local binarization, thesystem comprising: an area setting module for setting a firstbinarization area in a portion of a binarization target area of an imagein the image in which the recognition target object is displayed, and afirst coefficient calculation area including the first binarization areaand having an area wider than the first binarization area by apredetermined ratio; a coefficient determination module for determininga first binarization coefficient based on pixel values of pixelsincluded in the first coefficient calculation area; and a control modulefor performing binarization on the first binarization area through thedetermined first binarization coefficient, wherein the area settingmodule sets a second binarization area in a portion other than the firstbinarization area, and a second coefficient calculation area includingthe second binarization area and having an area wider than the secondbinarization area, the coefficient determination module determines asecond binarization coefficient based on pixel values of pixels includedin the second coefficient calculation area, and the control moduleperforms binarization on the second binarization area through thedetermined second binarization coefficient.
 8. The system according toclaim 7, wherein the coefficient determination module determines thefirst binarization coefficient between an average value of pixel valuesof pixels included in the first coefficient calculation area and amaximum value among the pixel values.
 9. The system according to claim7, wherein the area setting module sets the second binarization area andthe second coefficient calculation area of a size the same as that ofthe first binarization area and the first coefficient calculation area,respectively.
 10. The system according to claim 7, further comprising arecognition processing module for labeling the binarization target areaafter binarization of the binarization target area is completed,searching for a preset pattern according to a display characteristic ofthe recognition target object from a result of the labeling, andprocessing recognition on a recognition target object included in thesearched pattern.